VISUAL TRACKING METHOD BY COLOR INFORMATION 



BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to a method of visual tracking 
with use of color. More particularly, the invention relates to 
visual target tracking method applicable for various fields in 
such techniques as correcting important parts of recorded images 
for broadcasting stations, tracking human face in unmanned 
monitoring systems, tracking speakers for remote conference or 
remote conversation trough communication networks, setting 
face-contour for face recognition in security systems, tracking 
of a specified target, and so forth. 

The need for tracking or recognizing a target having a specific 
color is increased in various application fields like broadcasting, 
unmanned monitoring systems , security systems , remote conference 
trough communication networks, control of unmanned flying object, 
unmanned docking systems, etc. Considering the fact that ma j orit; 
of information human beings obtain is through visual means, the 
visual tracking techniques is expected to further expand their 
application fields in the future. 

Description of the Related Art 
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In conventional visual tracking techniques using color 
information, colors of the target have been represented by color 
models, which are known to be robust to illumination changes - 
such as in normalized Red-Green-Blue (R-G-B) space or 
Hue-Saturation-Intensity (H-S-I) space. 

However, these color models well accommodate uniform changes 
of illumination, but show limitations in the cases where the 
brightness of the target changes irregularly due to angle changes 
between light source and target or the brightness changes abruptly . 

For example, when a man is walking through a gallery whose 
ceiling is equipped with fluorescent lights at uniform intervals, 
the relative position of the immediately affecting fluorescent 
light to the man keeps changing and, therefore, the brightness 
of the man' s face keeps changing. And depending on the direction 
the man is moving, it often probable that one side of the face 
gets darken while the other side gets lighted. Moreover, surface 
reflectiveness of human face is hardly uniform due to secretion 
like sweat and it is extremely unlikely for the whole face to 
have uniform illumination changes. 

In other words, since color distribution changes a" 
illumination intensity varies, the conventional visual tracking 
techniques have problems in using a color model normalized and 
set-up for uniform illumination changes. 



SUMMARY OF THE INVENTION 



It is an object of the present invention to resolve the 
aforementioned problems by providing a method of visual tracking 
5 with use of color information which is robust to irregular or 

abrupt changes of illumination. 

It is another object of the present invention to provide a 
visual tracking method using color information in which 
characteristic of the photographing element of the camera is 
LO analyzed and the results of the analysis is modeled by B-spline 

curve, then allowing real-time visual tracking and application 
for the situations of rapid movement of the target. 

The present invention proposes a visual tracking method in 
which the photographing element of camera (CCD or CMOS) is analyzed 
L5 for its characteristics to the brightness and the analysis results 

are modeled beforehand so that it resolves the problems of the 
conventional visual tracking algorithm which is not adaptable 
to non-uniform or irregular change of illumination. 

The present invention also proposes a motion acceleration 
20 predictor for improving the speed of visual tracking. 

The present invention provides a visual tracking method using 
color information comprising: three-dimensional color modeling 
step in which images obtained under various illumination 
conditions are analyzed and thereby the photographing 
25 characteristics of the camera as to the target is represented 
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by a three-dimensional model; target recognition step in which 
judgement is made by the difference between the previous and the 
current images about whether or not a new target object appears, 
target region is located by applying the color model in said 
5 three-dimensional color modeling step, and the final decision 
is made as to whether or not visual tracking is to be performed 
for the target depending on the shape analysis of the target region • 
and the third step of visual tracking in which an arbitrary pixel 
is monitored and judged using said color model if it belongs to 
2jo the target region and at the same time the judgement process has 

to be adaptable to the movement speed of the target by estimating 
the movement of the target region. 

The above and other features and advantages of the present 
invention will be more clearly understood for those skilled in 
Ji.5 the art from the following detailed description taken in 

7 conjunction with the accompanying drawings, which form a part 

of this disclosure. 

BRIEF DESCRIPTION OF THE DRAWINGS 

20 

FIG. 1 is a schematic block diagram representing the concept 
of the visual tracking method according to the present invention; 

FIG. 2 is a graph showing the average hue to the brightness 
value modeled by the present invention; 
25 FIG. 3 is a graph showing the standard deviation of hue to 
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the brightness value modeled by the present invention; 

FIG. 4 is a graph showing the average saturation to the 
brightness value modeled by the present invention; and 

FIG. 5 is a graph showing the standard deviation of saturation 
5 to the brightness value modeled by the present invention. 

DETAILED DESCRIPTION OF THE EMBODIMENT 

Hereinafter, the present invention is described in detail 

10 by referring to the accompanying drawings. 

First of all, the visual tracking method according to the 
present invention comprise the step of three-dimensional color 
modeling of the target (S100), the step of target recognition 
for initially perceiving the target (S200) , and the step of visual 

15 tracking for repetitive visual tracking (S300) ; and the basic 

concept of the invention is schematically shown in FIG. 1. 

The first step of three-dimensional color modeling (SlOO 1 ! 
is the step for establishing a three-dimensional color model and 
has to be performed before initiation of the visual tracking. 

20 The above color model consists of four quadratic functions 

representing average and standard deviation of target hue with 
respect to brightness changes and average and standard deviation 
of target saturation. And these four quadratic functions 
constitute a single Gaussian function. 

25 For this purpose, the model is obtained by dividing the whole 
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region subject to brightness changes of the image into n regions 
with regular sizes and by approximating two adjacent regions using 
quadratic curves, accomplishing the model for the whole region. 

Here, each region is represented by a quadratic function such 
as Eqn. 1 along with boundary conditions of the quadratic curves 
represented by Eqns . 2-4. In other words, the model is built 
by the region curve (Eqn. 1) along with boundary value condition 
(Eqn. 2), continuity condition at the boundary (Eqn. 3), and second 
derivative condition for the region curve (Eqn. 4) as shown in 
the followings. 

[Eqn. 1] 

fj. (x) =a i -x 2 +b i -x+c 1 . 
[Eqn. 2] 

f x (xj =a 1 -x 1 2 +b r x i +c i =f (xi) , for i=l, 2 , ...n-1 . 

f i (x i+ i) =ai-x i+ i 2 +bi-x i+ i+Ci=f (x i+1 ) , for i = l, 2 , ...n-1 . 

[Eqn. 3] 

2-a 1 -x 1+ i+bi=2-a 1+ i-Xi + i+b 1+ i, f ori=l , 2 , ...n-1 . 

[Eqn. 4] 
ai=0 . 

Herein, x is the brightness of the concerning pixel, and f (x) 



is the average or standard deviation of the hue or the average 
or standard deviation of the saturation for the given brightness 



If the whole range of brightness is divided into 5, the 
relational equation for obtaining the coefficients of the 
quadratic curve for each region can be represented by a single 
matrix equation of Eqn. 5 as shown in the followings. 
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Herein, (a!, b x , cj for i=l,2,3, and 4 denote coefficients 
for quadratic curve representing each region. 

Based on four relational equations obtained above, 
three-dimensional color model is represented by the following 



x . 



[Eqn. 5] 




Eqns. 6 and 7; i.e. 



[Eqn. 6] 

H m (i) - T h • H a (i) < H 3D (i) < H m (/ )+ T h • H a (/') 

5 

[Eqn. 7] 

S m (j)-T h • S a (i) < S 3D (i) < S m (/')+ T h • S a {i) 

% Herein, H m [i) for 0 < / < 255 is the function of hue average, 

Jio H^ii) for 0 < / < 255 is the function of hue standard deviation, 

S m (f) for 0 < / < 255 is the function of saturation average, S^i/) 
for 0 < / < 255 is the function of saturation standard deviation, 
!L and H 3D (i) for 0 < / < 255 and S 3D (i) for 0 < / < 255 are the color 

model functions. 

215 In the second step of target recognition (S200), judgement 

is made about whether a new target object appears by the difference 
between the previous and the current images, target region is 
located by applying the color model newly proposed in the previous 
step, and the final decision is made as to whether or not visual 

20 tracking is to be performed for the target by shape analysis of 

the target region. 

The aforementioned shape analysis is performed by comparing 
the actual shape of the image in the target region against the 
outlined shape inputted beforehand. For example, if human face 

25 is tracked, egg shape is chosen as the reference for contour 
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comparison; if a ball is tracked, circular shape is chosen as 
comparison reference. 

In the third step of visual tracking (S300), the recognized 
target region is continuously tracked. 

In this step, an arbitrary pixel is monitored and judged 
continuously if it belongs the target region using the formulated 
color model, while target region movement is estimated and the 
judgement process has to be accommodate the movement speed of 
the target. 

Kalman filter technique might be considered for the movement 
prediction. However, it has a slow algorithmic speed due to its 
heavy arithmetic load and shows drawbacks that prompt adaptation 
is impossible for abrupt movement because it utilizes too much 
of the past information. 

To cope with this difficulty, the present invention employs 
a simple movement tracking method, which uses analysis results 
of the past three images as shown in Eqn. 8; i.e., 



[Eqn. 

P m U) = 



d[ d F~ ] ., F m U-\)-2*F m U-2)+F m U-Z) 



Herein, P m (i) is the predicted acceleration of the target 
region in the i-th image, F m (i) is the position of the target 
region in the i-th image, and At is the time increment between 
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the i-th and (i-l)-th images. 

Figs. 2 through 5 show four curves which are modeled for face 
tracking according to the present invention under various 
illumination conditions. 

In Figs. 2 through 5, the abscissa represents brightness value 
while the ordinate represents average and standard deviation of 
hue and those of saturation, respectively. 

According to the present invention, the brightness of the 
face changes depending on the relative position of the ceiling 
fluorescent lamp to the face, and the tracking works continuously 
even if illumination is abruptly darkened. 

As shown previously, the present invention allows adaptable 
performance of camera when a target with a specific color is tracked 
under irregular or non-uniform change of illumination. In other 
words, the camera photographing characteristics as to the target 
are three-dimensionally modeled by analyzing the images obtained 
under various brightness conditions, thus enhancing reliability 
against illumination changes, and a simple acceleration predictor 
with a light arithmetic burden is employed for adaptation for 
movement speed change of the target . So that the present invention 
has the effect which can be utilized in various application fields 
likebroadcasting, unmannedmonitoring systems , security systems, 
remote conference trough communication networks, control of 
unmanned flying object, unmanned docking systems, and so forth. 

Although the present invention has been described and 
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illustrated in connection with the specific embodiments, it will 
be apparent for those skilled in the art that various modifications 
and changes may be made without departing from the idea and scope 
of the present invention set forth in the appended claims. 
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